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This document provides materials concerning the 
evaluation of Minnesota's Early Childhood Family Education Programs. 
Part I summarizes. the findings and conclusions of evaluations 
conducted since the first six pilot programs began in 1975. Part II 
deals with the characteristics of successful^ high quality programs* 
and includes an instrument designed to assist school districts in 
self-devaluation of their programs. Criteria for assessing the quality 
of program components are delineated in the areas of community 
outreach and input, parent education, child centered activities, and 
program operation. Part III describes the rapid growth of the 
program. Accomplishments and concerns are noted in regard to the 
extent of participation, program administration, assistance to 
districts, advisory council representation and functions, 
interdistrict cooperation, facilities, and scheduling. Part IV 
suggests a plan for the collection of information needed to monitor 
and evaluate programs for program development and long-term 
evaluation purposes. A long-term evaluation strategy developed by 
Irving Lazar is described in the final section of the report. (RH) 
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EXECUTIVE SUMMARY 

Evaluation of Early CMldhood Family Education has been a concern and a oriority 
since the first six oilot oroqrams beqan in 1975. Part I summarizes the 
findings and conclusions of the variety of evaluations conducted since that 
time. Most of these evaluations were summative or formative in nature and 
oroduced findings which have assisted significantly in the subsequent 
develooment of the oroqram. 

Part II deals with the characteristics of successful quality oroqrams and 
includes an instrument desiqnpd to assist school districts in self evaluation of 
their oroqrams. Quality criteria are delineated for community outreach and 
•inout, the oarent education comoonent, child centered activities and oroqram 
ooeration. 

The raoid qrowth of the oroqram since 1983 when the oroqram beqan to move from 
qrant-fundinq with administration by the Council on Quality Education to state 
aid/local levy formula-fundinq and administration by Community Education is 
described in Part III. Accomolishments and concerns are noted in reqard to the 
extent of oarticioation, oroqram administration and assistance to districts, 
advisory council reoresentation and functions, interdistrict coooeratior, 
facilities and schedulinq. Enhanced outreach, extensive coordination with other 
community resources and schedulinq to accommodate more families are amonq the 
noteworthy accomolishments. If current rates of oarticioation continue 
throughout the year, the oroqram will either meet or exceed oarticioation 
figures orojected in the 1985 biennial budqet. 

Part IV suqqests a olan for the collection of information needed to monitor and 
evaluate oroqrams, both for oroqram develooment anri for lonq-term evaluation 
Durooses. 

A lonq-term evaluation strateqy develooed by Irvinq Lazar of Cornell University 
■s described in the final section of the reoort. Because Minnesota is the first 
state to attemot to evaluate the lonq-term effects of this soecific oroqram with 
such an extensive and broad oarticioant oooulation base, there are no 
tried-and-true instruments available to accomolish this task. Consequently, 
Lazar identifies the essential tasks and suqqests a oossible strateqy for 
beqinninq this effort. 

A ten-minute videotaoe has been oreoared to accomoany this study reoort. It 
enables the viewer to. quickly "visit" several oroqrams, and qet a" oicture of 
what oarents and children are actually doinq as they oarticioate in the oroqram. 
Severa oarents offer their oersonal evaluation of Early Childhood Family 
Education. Excerots from an interview with Irvinq Lazar summarize the benefits 
Of the oroqram and olans for future evaluation of the Quality and 
cost-effectiveness of the oroqram. 

Cooies of the 1/2" VHS videotaoe may be borrowed from the Leqislative Reference 
Library (612/296-3398) or Early Childhood Family Education, Community Education 
bection, Minnesota Deoartment of Education (612/296-8311). 
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A SUMMARY OF PAST EVALUATIONS OF 
EARLY CHILDHOOD FAMILY EDUCATION PROGRAMS 
ADMINISTERED BY THE COUNCIL ON QUALITY EDUCATION 

Introduction 

Evaluation of Early Childhood Family Education (ECFE) was a priority of the 
Council on Quality Education (CQE) since the first six pilot programs began in 
fiscal year 1975. A variety of evaluation and research specialists assisted 
the Council and its Advisory Task Force on ECFE in directing the focus of 
evaluation between fiscal years 1975 and 1984. These included: Daniel 
Stufflebeam of Western Michigan University; Virginia Shipmai of Educational 
Testing Service; David Weikart and Dennis Oeloria of Michigan's High/Scope 
Foundation; Paul Ookecki of Vanderbilt University; Donald Pierson of the 
Brookline Early Education Project; Duane Duffy cf the U.S. General Accounting 
Office; Michael Patton, Richard Weinberg, and Shirley Moore of the University 
of Minnesota; and Irving Lazar of Cornell University. 

Two major approaches to evaluation were discussed. Formative evaluation looks 
at issues related to program implementation, such as compliance with 
legislative intent, content and quality of services, numbers of participants, 
program costs, and participant satisfaction. Summative evaluation examines 
Issues related to program outcomes, such as changes in children, parents and 
families es a result of ECFE participation. The general consensus of all 
Involved was that formative evaluation was an immediate and ongoing priority 
to ensure that the programs were doing what they were intended to do. Of 
central importance for formative evaluation efforts was to see if the ECFE 
concept was workable in a variety of settings and if the programs could gain 
community acceptance and participation. 

Since summative evaluation looks at the outcomes of program participation, by 
definition it applies only to established progrsm-i. Therefore, prior to 
fiscal year 1979, CQE focused almost exclusively on formative evaluation. 
Several evaluation strategies since that time ha^'e looked at short-term 
outcomes of program participation. However, the most effective approach to 
examining outcomes of ECFE is likely to be a longitudinal strategy that 
gathers information on first-time participants, as well as a matched group of 
non-participants, and then follows both of these groups of parents and 
children over a period of 10-20 years. Data on both groups would be compared 
periodically to determine if any differences may be due to participation in 
Early Childhood Family Education. Long-term evaluations, however, cost 
significantly more than either formative evaluation or summative evaluation of 
short-term outcomes. Although CQE applied to a variety of federal sources and 
requested additional state monies to finance such an effort, it was unable to 
secure sufficient funds. 

In the following pages, conclusions from formative and summative (short-term 
outcomes only) evaluations conducted by CQE between 1974 and 1984 will be 
summarized by fiscal year. 

Evaluation, Fiscal Year 1975 

Six pilot programs were in their first year of operation during this fiscal 
year. Evaluation was directed at describing the implementation of ECFE in the 
six sites. The year-end report from the evaluator (Johnson, 1975) emphasized 
that the local programs had evolved during the year and provided a detailed 
description of each site. Recommendations to the Council included: diversity 
in ECFE services should be supported and encouraged; program costs should be 
analyzed; in-service training on outreach techniques to improve participation 



should be provided to local staff; communication between local staff and 
elementary school personnel should be increased; the organization of parent 
education curriculum needed strengthening; and the objectives for children's 
participation needed clarification. 

Evaluation. Fiscal Year 1976 

Twelve programs were operating during the 1975-76 school year, seven new 
programs and five continuing, (The request for continuation funds from one of 
the original six sites was not approved.) The year-end evaluation report 
(Oberaigner, 1976) provided extensive descriptive information about each of 
the local programs which was compiUi through a written questionnaire and the 
evaluation consultant's on-site visits. Results of a telephone survey of 
parents were also included in the report. 

Conclusions and recommendations Included the following: 

1. The qualification and competence of local staff persons comprised the 
single most important strength of the programs. The report noted that 
staff had devoted substantial amounts of extra time and shown exceptional 
resourcefulness in meeting unanticipated needs and in making local changes. 

2. The 12 programs had gathered momentum during the year and gained the 
community's acceptance. 

3. Methods of collecting data and keeping records for the local programs 
needed to be Improved. 

4. Stated objectives of the local programs lacked surficient specificity. It 
was also suggested that each program prepare a statement of Its philosophy 
to clarify the framework of Its activities and to promote more 
understanding of its purposes. 

5. CQE should revise the process of applying for new and continuation grants 
to reduce the time consumed In writing the application and to shorten the 
time It takes for start-up of a program. 

6. Since programs were operating mainly along the typical school calendar, 
the consultant recommended this be examined to ascertain if activities 
should be available at other times. 

7. The report recommended examining the time local staff spend In contact 
with parents and children out of their total paid time. 

8. Local sites should establish internal evaluation procedures to assure that 
they were responding to the needs of parents. 

Results of the telephone survey of a sample of parents, conducted by the 
Minnesota Center for Social Research, wert also reviewed in the consultant's 
report. Nearly all parents gave a very high rating to their respective 
program. It was found that, for many families, the initial attraction was a 
specific service (e.g., health screening) and the opportunity for their child 
to interact with other children. A substantial majority said they planned to 
continue participating. Finally, the responses showed that parents generally 
enjoyed learning more about parenting and felt their children benefited by the 
developmental experiences. 

- 3 - 
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Evaluation. Fiscal Year 1977 



Thirteen pilot programs were operating this year, as the Council had shifted 
one of its cost-effective/innovative grants programs to the ECFE catenory 
The Minnesota Center for Social Research (MCSR), directed by Dr. Michael 
Patton, was contracted for a detailed study of program implementation Teams 
of evaluators visited each of the 13 sites. On-site Interviews were conducted 
with participants, advisory council members, program staff, and other school 
personnel. Additionally the teams observed operations in each location and 
repeated the telephone survey of parents. The final report prefaces its 
comments by describing ECFE programming as "exemplary" and states " in 
our experience with demonstration projects, it is unusual to find 'programs 
where implementation so closely follows intent, and where program 
implementation has been so thorough, so intense, and so successful " 'Patton 
1977. p. 4) * ' 

Responses to interviews with staff and parents indicated that the major 
strengths of the parent education component of ECFE were; (l) a fundamental 
focus on improved parenting skills, better parent -child relationships, and 
integration of parents into the schools; (2) mutual support in parent groups; 

? parental feelings of isolation; (4) gaining new information about 
child development; and (5) emphasis on the importance of parent input 
Weaknesses were (1) the need to get more participation; (2) solving problems 
c , t^^^il^^L^" }° program sites; and (3) poor space and facilities. 
Early childhood education sessions were conducted concurrently with parent 
education services in most sites. Major strengths of this component were; 
(1) providing opportunities for the child to be with other children; (2) 
helping the child make the transition from an indivic-j-al relationship with the 
parent to group situations; and (3) helping pa-nts to improve their 
interaction with their child. 

Parents and staff were quest, oned on their feelings about the priority of 
funding for ECFE programs relative to other educational programs There was 
strong support for a high priority of funding for ECFE. Several distinct 
themes emerged In response to this Issue (Patton. 1977, pp. 43-44): 

^- Imi/J'^^/^^' y?'^ ^"^^ crucial. We must get to both narpnts .nH 

— durlna the critical formative years . Over half — of — aTT 

respondents specifically mentioned this argument. 

A school board member; "Our children are our |)1ggest asset If we can 
properly take care of them—from the day they're born, it starts 
immediately, not just when they get to school --they' 11 remain' our chief 
asset. It all stem? back to the early years in the home." 

2. The parent Is the child's first an d most important teacher. Tf yn .. want 
to affect children vou have to affect parents first . Qvpr a thirH nf tho 
respondents mentioned this rationale. 

"Schools can have little effect if the home environment isn't supportive." 
"Parents hold the key to long-term effects in children." 

3- ffl'^y Education is a p r evention program. Other nmar^m^ 

IDL-lo_:c. ompensate" or "rehabilltatP": this Droor.m is preventino 
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firoblemi. A fourth of the respondents specifically used the word 
"prevention" in explaining their support for these programs This 
argument is closely tied to the idea that the first five years are the 
crucial years developmental ly. If problems don't develop then, schools 
won t have to spend large sums of money trying to deal with those problems. 

M.'f"^^"^''°^?"'^ naturally: It's a .HI! th.t h.c be learned-and 
^hls argument °^ respondents specifically mentioned 

i'n!'olI,:r^„/*'"'^^"' ,9«"eration gaps, highly mobile families -the 
information on parenting is no longer handed down from generation to 
generation. The skills have to be learned some other way." 



5. 



Jhe isolation of people and families in modern society makes i t necessary 
to find new w ays for people to support each other . necessary 

^ntJ'if ^'*f"'^^"^ community. People don't know each other. Their own 
parents live far away or are deceased. They have no one to talk to about 
What IS happening with their child. Grandmother isn't there as par? of aJ 
extended family. In small families, children don't have siSlinS 
Playmates. This program is saving both mother and children from almost 
complete social isolation -it's a life support system building a new ens e 
of community among neighbors." a new sense 

During fiscal year 1977, CQE systematized its collection of figures on 
participation and cost across the programs. These annual sumr«ries docum^n? 

participaS?^::;: 

Evaluacio n. Fiscal Years 1978 and iQ7q 

Nine new programs began operating in the fall of 1978, bringing the total of 

llltltZ Plan that""'""id'° Task Forc'e deve ope a 

evaluation plan that would produce a series of related reports over these two 

Ii?»rho"„"f'-^t Research (MCSR) was again retained to conduct 

In ^t °::por\ (^It?on°'i9'7«r^ P^'-ticipating parents from each program 

p?evfL SLJs^ ^ir'nrnol^^' ""^1"^!. generally consistent with ?hose of 

previous years. The programs received an average rating of 3.7 on a scale of 

0-4, with zero representing low or undesirable ratings. Nearly all tJl 

respondents, 95 percent, said that funding for Early Childhood FamnS 

Education Should be viewed as either an "important" or "top'' prio??ty olll 

ad'^hXd "a'1reT"L;"'^"''?. P^^^icipatTon ''n ^[^e'pro ^a" 

naa neiped a great deal" or "somewhat." Responses centered on thp 

on tne value of the interaction. As with previous evaluations narents 
corrfliented on the high quality of local staff as an important strenaVh of 

Vo7Teir c;nd?e^''aVTh'^ ''''' ''''' '''''' for"/^ire%'rsVimu t 
D?Lram Mr<?p V? ^ ^"1® ^^^^ becoma involved in this 

L?il??;.. th.t''^''n?"J."^^'''^ "^^^^ ""^^^ P^'-^^ts initially are looking for 
? based on thp ct^'' '"!' "'" V"'"' their continued partic ipat 2n 

IS based on the stimulat:on they receive in the activities designed or 
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adults. The most frequently mentioned weakness of the local programs was the 
need to expand services, either in the number of people that can be 
accommodated or with respect to the length of time services are offered 



Since prior evaluations had indicated successful implementation of the concept 
of Early Childhood Family Education, and responses of parents had been very 
positive, members of the Council and the Advisory Task Force felt that this 
was an appropriate point to assess the quality of the services provided at the 
local sites. Contractual arrangements were again made to have the Minnesota 
Center for Social Research coordinate the difficult task of defining quality 
criteria or standards of excellence for Early Childhood Family Education 
programs. In the spring of 1978, Dr. Michael Patton, director of the Center, 
began work with staff from each of the local program sites, as well as members 
of the Council, Task Force, and CQE staff. After extensive revision a final 
list of criteria was adopted in the fall. 

In October and November, 1978, teams from MCSR conducted on-site reviews at 
each of the 22 programs and made detailed use of the criteria through a 
five-point scale, with one being the highest rating. The report (Patton et 
al., 1978) showed that statewide, the criteria of highest quality were- (l) 
providing positive reinforcement to parents; (2) coordination with other 
community services; (3) providing parents with opportunities fo" personal 
growth and development; (4) program organization; (5) providing appropriate 
materials and activities for children; and (6) providing learning 
opportunities for parents in the centers. 

The criteria with the lowest overall statewide ratings were: (1) reflection 
of parent and home environment in center based children's activities- (2) 
parent participation in selecting children's materiils and activities' (3) 
attractiveness of program environment; (4) advisory council decision-making 
role; and (5) parent involvement in program planning and implementation The 
report noted that the overall mean for each of these relatively lower quality 
criteria was still well above the 3.0 level which represents basic quality 
implementation. Thus, these criteria were ranked low only in comparison to 
the other criteria. Overall, the data indicated that the 22 programs were 
attaining high levels of program quality. 

The evaluation team identified several elements operating in the ECFE programs 
which n felt accounted for such high quality (Patton et al., 1978, pp. 39-42): 

1. Quality of Staff; Staff exhibited commitment anjl enthusiasm, along with a 
demonstrated ability to operate their programs. 

2. Clarity of Purpose? The programs were clearly directed at improving 
parenting and were not offering therapy, nursery school, day care or 
social work support. ' 

3. Staff Colleaqueship : There was staff consensus about program content and 
processes, and separate service components were integrated into a smoothly 
operating whole. ' 

Sensitivity to Local Conditions : The programs combined sensitivity to 
local needs with leadership that provided a direction for improving local 
conditions. 



5- Community — Support: Local staff carefully integrated their program 
services with those of other community organizations. 

6. Program Autonomy: while the Legislature and CQE provided basic guidelines 
for program operation, the day-to-day content and processes of the 
programs were determined by local people to meet the needs of families in 
local communities. 

Two additional issues were addressed at length in the report. Parents who had 
been long-term participants were said to be a major untapped resource in most 
programs. It was suggested that these "experienced" parents take more active 
roles sucn as acting as home visitors to isolated or special needs families, 
laading selected parent groups in the center, and producing program 
newsletters. Program staff could then spend more time individualizing program 
services to better meet the diverse and complex needs of families in general, 
and particularly of parents from special groups with special needs. Secondly,' 
the report suggested that staff in newer programs located in isolated rural 
areas be paired in a "buddy system" with staff of well-established programs to 
facilitate their professional development, in general, it was suggested that 
CQE arrange more opportunities for programs statewide to share their resources 
with each other. 

In Hay of 1979, a mail survey of all 76 kindergarten teachers from the 49 
schools served by the 22 Early Childhood Family Education programs was 
administered. The return rate was 62 percent, which is slightly above average 
for a mail survey of this kind. Seventy percent of the responding 
kindergarten teachers had taught at least three or more children who had 
previously been part of a CQE early childhood and family education program 
Responses were reported only for teachers who knew .hey had taught at least 
one former program participant. 

Compared to kindergarten children in general, these teachers believe that 
children from CQE Early Childhood Family Education programs have more positive 
attitudes toward school (90 percent agree); better all-around preparation for 
school (92 percent agree); better preparation in pre-kindsrgarten basic skills 
(87 percent agree); more confidence (92 percent agree); more social skills for 
interacting with other students (90 percent agree); better relationships with 
their parents (86 percent agree); fewer behavior problems (77 percent agree); 
and more emotional maturity (79 percent agree). 

Teachers were also asked about their perceptions concerning the impact of the 
ECFE programs on parents. Compared to parents in general, kindergarten 
teachers believe that parents who have participated in Early Childhood Family 
Education programs are more active in their child's school (87 percent agree)- 
more knowledgeable about child development (87 percent agree); more able to 
help their children with their school experience (87 percent agree)- more 
confident as parents (87 percent agree); and more communicative with' early 
elementary teachers (87 percent). 

percent of the teachers responding felt that funding for Early 
Childhood Family Education should be given either a hiah (70 percent) or 
medium level (26 percent) priority by the Minnesota Legislature. It is clear 
from these responses that kindergarten teachers believe that tarly Childhood 

nfrlntc positive and visible impact on participating 

parents and children. (Patton, 1979) k v y 

- 7 - 



ERIC 



10 



Evaluation, Fiscal Years 1980 and 1981 

Again in 1979. the Minnesota Legislature increased the appropriation for Early 
Childhood Family Education and stated that up to 36 programs could be operated 
with these funds. New proposals were solicited, and the Council transmitted 
grants to 13 new programs in the fall of 1979. One of the third yea- programs 
was split into two separate programs, bringing the total of locations to 36. 

During the spring and summer 1979. Dr. Michael Patton worked with the 
directors of "established" local programs to develop a questionnaire designed 
for completion by parents upon beginning and again upon completing a phase of 
program participation. The questionnaire was field tested in 20 of the 
program locations and completed by over 2.240 parents during the fall of 1979. 

Analysis of the results of this field test Indicated that the parent 
self-assessment questionnaire basically failed as a pre-test to POSt-test 
instrument Especially on the behavior, personal opinion, and knowledge 
items there was little room for movement as a result of progrdm 
participation. Respondents generally answered these items nea'* the desired 
extreme of the continuum on the pre-test. Either parents enter the programs 
already exemplifying the qualities considered most desirable by the program 
directors, or program impact is in areas of nuance and increased confidence 
and certainty in desirable parental qualities. If the latter is the case. Or. 
Patton speculates that possibly no standardized instrument will be abl- to 
pick up on these subtle nuances, and that a more qualitative research approach 
will be required to measure program impact. (Patton. 1979) 

During the summer and fall of 1980. the firm of Anderson & Berdie Associates. 
Inc was retained to look at short-term outcomes o; Early Childhood Family 
Education programs in either their sixth or seventh year of operation. It was 
initially decided that one rural, one suburban, and one city program would be 
examined. During the early phase of tha evaluation, however, it was 
determined thar neither the Minneapolis nor St. Paul programs could be 
effectively evaluated within the specified design due to the geographic 
mobility of participant families. Therefore, an increased sample size was 
drawn from tha rural (Staples Rural Family Development Program) and suburbon 
(Bloomington Parent & Child: Growing Two-gether Program) sites selected for 
the study. The purpose of the evaluation was to determine whether or not 
these two programs had an impact upon their participants. 

Three techniques were used to gather information for the study. First, 
parents who participated in the ECFE programs and parents who did not 
participate in these programs were surveyed by telephone. Second, teachers of 
kindergarten classes which contained a significant number of students who had 
formerly participated in ECFE were asked to rate all children in their classes 
on dimensions addressed by the programs. Third, independent observers rated 
these classes of children using the same dimensions as the teachers had used. 

In reporting the results, the authors caution that generalizing the findings 
of this evaluation project to other ECFE programs located throughout the State 
may or may not be accurate, since the differences amor.g the programs make such 
generalizations difficult. Additionally, the design limitations on the study 
did not allow for measures on the parents and children who participated in the 
Bloomington and Staples study prior to their participation. This presents two 
significant methodological problems. First, it cannot be assumed that the 
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parents who participated in the ECFE programs are a random sample of parents* 
Thus, differences observed between program participants and nonparticipants 
may or may not be attributable to the program* Second, without preliminary 
data on participants, measuring impact of the program on parental confidence 
or skill in the classic research sense is nearly impossible* Yet since this 
evaluation approach included three types of data collection - parental 
comparisons, child comparisons, and parent opinions, valid conclusions can 
still be drawn* These are summarized below* (Anderson & Berdie, 1980. pp. 
19-21) 

1* Parent participants and nonparticipants at Staples did not differ in many 
ways, while there were notable differences between participants and 
nonparticipants at Bloomington. One reason for this may be that 
nonparticipants in Staples rated higher in certain desirable parenting 
behaviors than did Bloomington nonparticipants. This might allow more 
evidence of program impact on parents to emerge in the Bloomington program* 

2. There were no significant differences between teacher or independent 
observing ratings of participant and nonparticipant children in aither the 
Bloomington or Staples programs. However, three factors may explain this 
lack of evident impact* First, teachers rated most of their students 




, pick up 

differences that might exist between participant and nonparticipant 
Chi Idren. 

Both the Staples and Bloomington programs looked very favorable when 
assessed by directly asking participants about the impact cf ECFE* All of 
the participants at both sites say that since their participation in the 
ECFE program in their area, they: (a) are better informed as a parent, 

(b) are more aware that other parents have many of the same problems, and 

(c) have a better understanding of the similarities and differences among 
children* Virtually all (more than 90 percent) of participants at both 

C^toC caw that tho%i fi/x.-i. /^\ -.^^ ^ i r ^ _ ^ 



The quality criteria for Early Childhood Family Education programs developed 
by Or* Michael Patton and local program directors (discussed earlier) were 
revised in the fall of 1980. Ors. Karen Hess and John Bowers directed this 
evaluation effort. Their tasks were to assess the quality of 13 second-year 
ECFE programs and to identify potential areas for improvement in each of the 
programs. 

The evaluators began with an intensive document review on the 13 programs, 
including the second-year proposal and negotiated grant budget for each* 
Evaluation instruments (including the Patton quality criteria) were then 
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revised and/or developed with extensive input from the program directors. 
After completion of this process, Ors. Hess and Bowers visited the programs 
separately, so that each program received an initial and a follow-up visit. 
In addition to the site visits, participating parents and advisory council 
members were surveyed on their reactions to and assessments of their local 
program. 

As the final report (Hess & Bowers, 19B0) describes, the 13 second-year 
programs were very diverse in geography, economics, and ethnicity. Five were 
in metro*. ^n areas and eight were in small towns or sparsely populated 
rural are^n Of the latter eight programs, three were operating on American 
Indian reservations. Although the types of service components p»"ovided are 
simi lar, the manner in which they were offered varies . The budgets and 
staffing of the programs also varied considerably. 

Hess and Bowers summarized the central conclusion of their report as follows: 

The main conclusion of this study can be s.uiply stated: thirteen 
second-year ECFE programs funded by the Minnesota Council on Quality 
Education have been effectively implemented in diverse settings and have 
positively benefited parents and children in ways reflecting CQE's and 
their individual intended objectives. . . . Programs are characterized by 
clarity of mission, productive activity, creative direction, staff 
competency, and flexible adaptation to local needs of parents and their 
children. (Hess and Bowers, 19B0, p. 31) 

In reporting on the results of the parents' survey. Hess and Bowers describe 
the parent participants in the ECFE programs^ ns unquestionably "true 
believers". 

Parent ratings of program features were exceptionally high. Program staff 
and materials and activities provided for children were highest rated of 
various program features. Program opportunities provided parents to 
share problems, to make suggestions, to feel good as a parent, to discuss 
matters of importance to them, and to build on th^air strengths as parents 
- received high ratings from parents. Parent activities and materials 
received slightly lower average ratings, but they were, nevertheless, 
high. (Hess and Bowers, 1980, p. 33) 

Other recommendations included the following (Hess & Bowers, 19B0, pp. 34-36): 

1. Th3 role of advisory councils may not be altogether satisfactory in some 
cases for these programs. Each program should have an ECFE advisory 
council that is fx)rmed for the program; existing advisory groups should 
not assume "add-on" ECFE responsibilities. That is the surest way to run 
a risk of having an ECFE program co-opted for other than intended 
purposes. In addition, advisory councils should include a good 
cross -section of the population served, and members should have their 
roles on the counci^ clearly defined. An active, effective advisory 
council can do much to stretch the resources available to an ECFE program, 

2. ECFE programs should be coordinated with the district and, if such exists, 
its community education division. In the sites visited, districts for 
these programs were not only highly supportive, but appeared to maintain a 
separation of ECFE and district objectives. Schools are and should be 
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represented in programs, but their influence must not overtip ECFE program 
objectives. 

3. Continuing in-service is needed by some programs. Directors are 
appreciative of CQE support given to program staff development and regard 
its effects highly. In-service is costly; time for it must be efficiently 
used. Agendas devoted to specific program components would be of benefit 
to less experienced directors who want ideas. Program staff also request 
opoortunities for more cross-program exchange. 

4. The Council's ECFL program is an outstanding example of effective program 
development and support. Program concepts are clear; parent and chi Id 
activities are developed in response to local needs. There is no doubt 
that these programs also have significant effects on parents and 
children. Effects are not currently measurable. They could be. And the 
public should be told of them when they are. 

5. Second-year programs, certainly those represented here, are capable of 
carrying out self -evaluation, but procedures for doing so efficiently are 
needed. An excellent step taken by the Council was its development of 
criteria and indicators of the quality of program components. A11 program 
directors should be introduced to these quality crUeria early and 
reminded of them often. Self -assessment procedures and progress measures 
related to ECFE criteria should be developed and reported from the 
Inception of any program. 

In the far of 1980, an evaluation team began collecting information for case 
studies of a sample of families beginning particioation in three program 
locations - Fairmont, Ouluth, and Minneapolis. The evaluators were to look 
for potential changes resulting from involvement in the program over the 
course of one school year. 

The following seven dimensions formed the focus of the study (List, Reiner, 
and La Freniere, 1981): parental self-concept; parental involvement in child 
care; family support systems; parental discipline a'lv control; parental 
nurturance and responsiveness; parental awareness of complexity in child 
development; and parental guidance of child behavior. Multiple methods were 
developed to maximize the extent to which potential outcomes of program 
participation would be detected: 1) semi-structured interviews; 2) 
naturalistic observations in the home and cent'jr; 3) videotaping parent-child 
interaction while engaged in se* -naturalistic situations; and 4) 
questionnaires and assessment instruments. 

Thirteen families initially were selected to participate in the evaluation 
project. An attempt was made to obtain as diverse a sample of families as 
possible. Ten families were visited in the fall and spring. The remaining 
three families were visited only in the fall; at various times, these families 
discontinued program participation. These families were contacted in the 
spring to determine their reasons for dropping out of the program. A detailed 
case study on each family is contained in the body of the report. 

Perhaps the most striking finding from this evaluation was the di verify of 
outcomes potentially attributable to program participation. However, several 
consistent trends were identified across the families. From an examination of 
the case studies as a whole, the evaluators drew the following conclusions. 
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First, little change was observed on the following dimensions of family life: 
1) parental nurturance and responsiveness; 2) parental involvement in child 
care; and 3) family support systems* In contrast, changes were consistently 
found in the following areas of family functioning that appeared attributable 
to program participation: 1) parental self -concept; 2) parental discipline 
and control; 3) parental awareness of complexity in child development; and 4) 
parental guidance of child behavior. In addition, parents often mentioned 
benefits of program involvement for their children. 

Finally, the report stresses that it is necessary co qualify the findings of 
this evaluation. First, the project was exploratory in nature. The results 
should not be viewed as a definitive outcome evaluation of the ECFE program. 
Second, the voluntary nature of the sample selection procedure may limit the 
scope of the findings. Third, jiven time constraints in implementing this 
evaluation, some program-related changes may even have occurred prior to the 
initial fall visit. However, this report documents several discernible and 
potential outcomes in family life as a result of participation in ECFE 
programs. 

Evaluation. Fiscal Years 1982 to 1984 

Fiscal cutbacks, reductions in CQE staff, and legislative changes for the role 
of CQE with ECFE limited the scope of evaluation efforts during these years. 
Collection of annual participant/cost data continued, however. These show 
that in fiscal 1982, the 34 ECFE programs (two programs in Bloomington and in 
Minneapolis both merged into single programs, reducing the total of pilot 
sites from 36 to 34) served slightly less than 18,000 parents and children two 
or more times. Approximately 10,000 participants were involved ten or more 
times. Support for each program averaged $41,000 in CQE grants, with another 
$28,000 derived from other funding sources. Total support for the programs 
averaged $69,000, but ranged substantially from $30,000 to $213,000. 

Overall costs averaged $130 per participant involved two or more times. Of 
this, 64 percent was paid with CQE grant funds. The remainder came from other 
sources, including vocational education, community education, special 
education, parent fees, and local district contributions. Average costs for 
participants involved ten or more times were $234, reflecting the same 
proportions . 

In the fall of 1982, Ors. Richard Weinberg and Judith Brady were contracted to 
produce a blueprint for future evaluation of ECFE programs coordinated by 
CQE. The report (Weinberg & Brady, 1983) points out that there has been a 
shift in the focus of evaluation of early childhood programs across the 
nation. Due to the vaiue placed on pluralism and local initiative, evaluation 
is having to accommodate an increasing diversity in programming. Evaluators 
are less able to offer a single plan for program assessment, but must suggest 
a set of alternative strategies to document program implementation and impact. 

The report states that there will always be a need for formative evaluation to 
insure that local ECFE programs are adequately meeting the needs of their 
communities. Summative evaluation could focus on the commonalities among 
local programs, such as certain skills which all programs agree are necessary 
for successful child-rearing. For example,, the National Committee for 
Prevention of Child Abuse recomrriends that all parents learn to develop 
networks of social support and to locate and make use of appropriate 
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information a: their children develop. 

Four recommendations were presented for future evaluation activities (Weinberg 
& Brady, 1983): 

1. Create a data collection system and bank on program participants. 

2. establish an in-service program for ECFE program directors to share 
perspectives on evaluation and useful strategies. 

3. Establish the role of ECFE evaluation/research coordinator to provide 
liaison among CQE programs, central staff, and college and university 
training programs and to facilitate a network of ongoing student and 
faculty-initiated evaluation activities within the programs. 

4. Maintain and update a list of ECFE-related questions or topics which might 
be studied in particular programs or clusters of programs. 

Summary 

The preceding pages have summarized the conclusions from Council on Quality 
Education evaluations of pilot Early Childhood Family Education programs since 
their inception in 1974. Evaluation tasks have been performed by individual 
consultants, outside agencies, CQE staff, and local program personnel. 

Over time, the findings have reflected a steady growth in the quality of 
program services and administration. However, a variety of areas that needed 
improvement were also identified, proving most useful to CQE in program 
coordination statewide. The most consistent evaluation findings document the 
competence and commitment of staff in the local programs and the favorable 
attitudes of participating parents. There has also been substantial agreement 
among evaluators about the strength of the program and its concept. 

A recently published book, entitled Strengthening Fami lies . describes parent 
education and child care as strategies critical to maximizing human 
development in this country. In a chapter entitled "Exemplary State 
Programs," Minnesota's Early Childhood Family Education program is described 
at length as a model of parent education for the nation. The authors 
acknowledge that it will be a challenge to show with hard data that ECFE 
programs work and are cost-effective. 

Many evaluation experts, however, would argue that th^ Council has already 
generated the evidence most relevant to policy decisions: The overall 
program is operating according to budget and specifications, and staff, 
parents, and relevant community members rate it highly. Measurable 
behavioral outcomes for parents and children will be difficult to 
demonstrate in the short run, given the state of the art of measurement 
and program evaluation. Moreover, ECFE was designed to provide readily 
accessible resources to all families within a program community; it is not 
intended to be an intensive long-term intervention for a few problem 
families. Even in the latter instance, the demand for measurable effects 
would push the limits of program evaluation technology. Given the broad 
goals of the program, evidence of accessibility, use, and parents' 
satisfaction perhaps is all that is possible and all that should be 
required. That evidence is available, and it is uniformly positive. 
(Hobbs et al., 1984, p. 267) 
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CHARACTERISTICS OF SUCCESSFUL PROGRAMS 
AN INSTRUMENT TO ASSIST DISTRICTS IN SELF EVALUATION OF PROGRAMS 



Bundlnq uoon the results of earlier evaluations described In Part I, and 
conslderlnq chanqes In Early Childhood Family Education leqlslatlon In and since 
1984, the followinq Instrument has been develooed to assist local oroqrams In 
the evaluation of their oroqrams. It Is also used by state staff and will 
become a oart of the Minnesota Community Education Association's oroqram revlpw 
orocess. The oroqram comoonent criteria listed reflect the characteristics of 
successful oroqrams and Indlcatp comol lance with current Minnesota Statutes and 
oroDOsed Board of Education Proqram Rules. 

Evaluation Is an on-qo1nq Inservlce toolc for oroqram coordinators/directors. 
At a forthcominq meetinq of all oroqram coordinators, this evaluation Instrumpnt 
and Dlans for lonq-term oroqram evaluation, with the accomoanylnq data 
collection needs, will be the ma.ior focus. Dr. Irvinq Lazar of Cornell 
university, who has assisted In the develooment of a lonq-term evaluation olan 
described later In this reoort, will be oresent to clarify and resoond to 
questions on evaluation strateqies for the Early Childhood Family Education 
oroqram both locally and statewide. Local oroqram staff and advisory councils 
W ( be requested to evaluate their own oroqram with the followinq Instrument 
onor to this meetinq. 
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EARLY CHILDHOOD FAMILY EDUCATION 
PROGRAM EVALUATION 



Program Component 
Criteria 



Rate each item 1-5 
as indicated 



5 4 3 2 1 
Exc. Good Avg. Fair Poor 




B. 
C. 



COMMUNITY OUTREACH AND INPUT 
A. Extensive personal outreach is made to 
parents with young children from birth 
to kindergarten enrollment. 
The program is planned to meet the speci- 
fic needs of the area served. 
The Advisory Council plays cn active 
decision-making role in program planning, 
development and monitoring. 
The Advisory Council membership includes 
a variety of community and parent repre- 
sentatives. 

Parents are actively involved in program 
planning, development and monitoring. 
The program adequately represents all 
socio-economic groups in the community. 
Arrangements are made to facilitate parent 
participation (flexible hours, transporta- 
tion, convenient sites). 
A variety of activities is offered to 
accommodate different needs and learning 
styles. 

The early childhood family education pro- 
gram coordinates with other community 
servic »s and resources. 

The program appropriately supplements, but 
does not supplant, the functions or mandates 
of other programs. 

The program is considered an integral part 
of the school district and has its support, 

COMMENTS: 



E. 
F. 
G. 

H. 
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EARLY CHILDHOOD FAMILY EDUCATION 
PROGRAM EVALUATION 



Program Component 
Criteria 



Rate each item U5 5 4 

as indicated Exc. Good 



B 



C. 



D. 



II. PARENT EDUCATION 

A. The principles of child development and 
parent education are taught by using 
parents* own concrete experiences. 
Parents are positively reinforced to 
affirm and enhance their positive 
feelings about themselves and their 
parent role. 

The parent educator possesses the 
knowledge and ability to infuse new in^ 
formation and facilitate parents' learn- 
ing and growing through group processes. 
The program nurtures and enhances the 
healthy functioning of the variety of 
family systems represented by emphasizing 
and providing opportunities for personal 
growth and on-going adult development. 
Educational materials on early childhood 
and parent education available as resour- 
ces are reviewed for sexual, racial and 
cultural bias. 

Parents who partit ipate in center-based 
programs have opportunities to learn and 
grew through group discussions. 
Home visits are carefully planned to meet 
specific parent/child needs. 



COMMENTS: 



E. 



F. 



G. 



3 

Avg. 



2 1 
Fair Poor 
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EARLY CHILDHOOD FAMILY EDUCATION 
PROGRAM EVALUATION 



Program Component Rate each Item 1-5 5 4 3 2 1 
Criteria as indicated Exc. Good Avg. Fair Poor 


III. CHILD-CENTERED ACTIVITIES 

A. Child development activities and exper- 
iences are designed and directed to 
promote child development and build 
interest. 

B. Learning materials and activities for 
children are appropriate to the child's 
developmental level, needs and interests. 

C. Parents and children have choices in 
selecting materials and activities in 
which they will participate, 

D. Parents are involved in actual child 
development activities as prime educa- 
tors of their children. 

E. Parents are provided information on the 
purposes of their children's activities. 

F. Center-based child development activi- 
ties reflect the importance of the 
parent and the home environment. 

G. The center environment is safe, interes- 
ting and stimulating to children. 

COMMENTS: 
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EARLY CHILDHOOD FAMILY EDUCATION 
PROGRAM EVALUATION 



Program Component Rate each item 1-5 5 4 3 2 1 
Criteria as indicated Exc. Good Ava. Fair Poor 


IV. PROGRAM OPERATIONS 

A. The program environment is attractive, 
inviting, and personal for parents and 
children. 

B. Professional staff members are competent 
family educators and are appropriately 
licensed. 

C. Professional staff are paid a professional 
wage and provided staff development oppor- 
tunities. 

D. Staff work together well, form a congruent 
and integrated professional group. 

E. The program is well organized. 

F. The program has clearly defined, relevant 
objectives. 

G. The program's clear purpose is education, 
not therapy, nursery school or day care. 

H. The staff conducts evaluations of the 
program. 

I. Records are maintained to document prog im 
participation and program activities. 

J. Fees, if charged, are reasonable and 

easily waived for families unable to pay. 

K. A program budget which accounts for all 
Early Childhood Family Education aid, 
levy and other funds is on file with the 
program coordinator and the district 
administrator. 

COMMENTS: 
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EARLY CHIL6H00D FAMILY EDUCATION 
PROGRAM EVALUATION 



V. Component Summary 



A. Community Outreach and Input 

B. Parent Education 

C. Child-Centered Activities 

D. Projiram Operations 



Total by N Mean 
ScQye Rating 



-r 11 

-f. by 8 

by 7 

^ by II 



Criteria developed by M. Patton et al. , 1978 
Revised by K* Hess and J. Bowers, 1980 
Revised by L. Engstrom, 1985 




PROGRAM GROWTH 
ACCOMPLISHMENTS AND CONCERNS 



In resoonse to the leqislatlon of 1984 which established a stat(» aid/local levy 
formula for funding Early Childhood Family Education, 253 school districts 
levied In the fall of 1984 for a oroqram to be Imolemented In the 1985-86 school 
year. 

The 253 districts Involved collectively levied fil« of their levy limit. As of 
December 1985, 93« of these districts had a oroqram In ooeratlon. The remalnlnq 
7% reoorted that they were still In the olannlnq staqes; had exoerlPncPd ^- 
chanqe In district administration that affected oroaress; could not find 
qualified staff or were concentratinq on hulldinq a base of suooort for the 
oroqram. Seventy- two oercent of oroqrams statewide were within their first two 
years of oroqram Imolementatlon. The other 281S either had established a oroqram 
earlier with Council on Ouality Education (COE) qrant fundinq or throuah 
Commun. ty Education with minimal fundinq from a variety of sources. 

The number of districts currently involved in Early Childhood Family Education 
reoresents substantial qrowth of aooroximatel y 800« over the 34 COE sites 
located in 29 districts in 1982-83. Durinq this exoansion staqe, exoerienced 
oroqrams have been Invaluable in orovidinq assistance to new oroqrams. 
Conversely, new oroqrams have been leaminq new techniques, esoecially in the 
areas of outreach and coordination with other community resources, that are 
beneficial to exoerienced oroqrams. 

Sharinq this workinq knowledqe has been a major ^^rust of the Deoartment of 
Education. In January of 1984, a 268 oaae Guide for HeveloDinq Early Childhood 
Family Edu cation Programs was disseminated to all school districts. The quide 
is a comoiiation of ideas and information gathered from all existing orograms at 
that time in resoonse to the question, "What do you know now that vou wish you 
had known when your orogram was just beginning?" A second major effort was "the 
establishment of a statewide regional inservice network to orovide a minimum of 
three days of concentrated inservice education oer year for Early Childhood 
Family Education staff, administrators and advisory council members. Two 
volunteers from each of the eleven regions oarticioate in a train-the-trainer 
session facilitated by state staff; they then facilitate a similar session in 
tneir own region using materials and resources orovided or arranged for at the 
training session. This system orovides for easily accessible inservice sessions 
where new information can be disseminated and discussed throughout the state 
within a 30 day oeriod; oartlcioants have an oooortunity to share information 
?Mi^u°"?''r*'.J''*''?l minimized; the leadershio base for Early 

rT.ltii°A r 1^ Education is greatly enhanced and an invaluable networking 
caoability is being created. More than 400 oersons have oarticioated in each of 
the three series of regional sessions conducted within the oast year. When 
districts were asked to indicate which kinds of deoartment assistance or 
re-.ources they had used, the regional inservice sessions and the Guide were each 
listed by 84% of the resoondents. Other forms of assistance SSFTfreguently 
indicated were: 

Current lic€:isure information - 86« 

Questions and Answers on Early Childhood Family Education - 71% 

General uodate mailings - 73% 

Newsletter entitled Partners in Learning - 69X 

(A ioint venture of Early Childhood Family Education and Vocational 

Consumer and Family Education) 

23 
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Early Childhood Family Education workshoos at major conferences - 65« 
Someone who could resoond to soecific questions or concerns was listen! as 
the most critical and most aooreciated resource. 

Administration by the Deoartment of Education has also included the following 
efforts to assist districts in develooinq quality oroqrams: 

1) pevelooment of orooosed oroqram rules, currently under consideration bv 
the State Board of Education. 

2) Study cf aoorooriate teacher licenses and subsequent rieveloament of two 
new licenses - Parent Educator and Early Childhood Family Educator - now 
beinq studied by the Board of Teachinq. 

°^ health, safety and educational standards for the 
children's learninq environment. 

4) Comoilation and dissemination of models for coordinatinq Early Childhood 
Family Education and Early Childhood Soecial Education. 

5) Enhanced coordination with vocational education, early childhood 
screeninq, oublic health nurses, oublic libraries, child care oroviders, 
mnnesota Early Learninq Oesiqn, MN Council on Children. Youth, and 
Families and others. Local coordination efforts likewise have been qreatly 
increased with oroqram administration throuqh Community Education. 

6) Extensive collaboration with reoresentatives of colleqes, universities 
and vocational adult extension enqaqed in tf^acher oreoaration and/or 
inservice education. 

7) Develooment of a one credit course or workshoo entitled "Introduction to 
Early Childhood Family Education" - to be required of all instructional and 
administrative staff. Forty-five hiqher education reoresentatives have 
already been trained to teach the course throuqhout the state. Additional 
traininq sessions will be orovided in the future. A comolete traininq 
manual has been develooed in conjunction with hiqher education to helo 
assure a deqree of consistency throuqhout the state. 



The followinq is a summary of information derived from a survey of the 253 
levyinq districts, based uoon oroqram activity durinq the oeriod of July 1 to 

December 30, 1985. For most districts, this reoresents less than one-half of 
their oroqram year. 

EXTENT OF PARTICIPATION 



Parent Child Classes 
(6-12 sessions) 

Soecial Events 
(one-time activity) 

Home Visits 



Parents 
23,269 

57, 852 

916 



Children 
23,302 

39,586 

985 



Siblinqs 
5,650 

14,489 

347 



240,400 different families received newsletters from their local oroqram. 
2,760 oarents served as members of an advisory council or task force. 
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If the current rate of oarticioation continues throughout the remainrlf^r of thp 
year, the oroqram statewide wm either meet 0/ exceed oro.iectprt figures 
Included In the biennial budget of 1985* 

An on-going concern Is that of reaching famlHtrS who are not likely to walk in 
and register In resocnse to the usual oubilc relations or marketing efforts. 
Great strides are being made by a growing number of local orograms, but 
statewide, minority and disadvantaged families are still seriously 
under-reoresented the orogram. One regional Inservlce network series focused 
uoon outreach tecnn1oi;es and coordination with other community resources to 
better meet the needs of these families. The sharing of both successful and 
unsuccessful exoerlences was extremely useful and encouraging. Although the 
effort exoended Is not quickly reflected by oartldoatlon numbers, orogram staff 
recoqnize that while all families benefit from such a oroqram, disadvantaqed 
families usually make the qreatest qalns, and therefore, reachinq them Is a hlqh 
orlorlty. The Minnesota Leqislature which established a fundinq formula based 
uoon the number of very younq children In the school district, rather than uoon 
enrolled oartldoants. Is to be commended for Its foreslqht In this area. 



ADVISORY COUNCILS 

In addition to the required majority of oartlcloatinq oarents, which Included 
emoloyed and non-emoloyed, sinqle, steo, and foster oarents, reoresyntatlves on 
the Early Childhood Family Education Advisory Council most frequently mentioned 
include: 

Child Care Providers Vocational Education 

Human Services Law Enforcement 

Ciepqy School Board Members 

Head Start School Counselors and 

Elementary Princioals Psvcholoqists 
Kinderqarten Teachers Aariculture Extension 

Health Professionals Senior Citizens 

Public Librarians Local Businesses 

Nursery School Directors Community Education 

Soecial Education Directors 
Private School Administrators WIC Proqram 

Home Economics Teachers 

The functions of the councils, indicated in order of frequency, are to: 

1) Serve as eyes and ears for the nroqram in the community; 

2) Assist with outreach; 

3) Monitor the oroqram; 

4) Conduct oublic relations efforts; 

5) Plar the oroqran^*, and 

6) Helo coordinate oroqram services with other community resources. 

♦Initial olanninq and desiqninq of the oroqram was often accomolished bv an Early 
Childhood Family Education Task Force which oreceded the Early Childhood Family 
Education Advisory Council. In most cases, at least some of the advisory counci 1 
members had been oart of the task force. 
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INTERDISTRICT COOPERATION 



Forty-one oercent of all dUtHcts indicated involvement in a coooerative arranqement 
with neiqhborinq school districts for Early Childhood Family Education. 

Several consortia or coooeratives of 6-8 school districts share coordination 

functions as well as staff and resources. A larqer number of districts share a 

coordinator or other staff with one or two neiqhborinq districts. Some districts 

ourchase oroqram services from an adjacent district for their families in lieu of 
establishinq their own oroqram. 

FACILITIES/SPACE 

Sixty-nine oercent of districts resoondinq indicated that: 

1) Adequate, aoorooriate soace was made available by the district, or 

V modified at moderate eviense to accommodate the 

Early Childhood Family Education oroqram. 

Others are borrowinq or rentinq soace in the community. 
SCHEDULING OF ACTIVITIES 

Proqrams reoort th«^. they are offerinq activities durino weekday morninqs, early 
afternoons, late afternoons, over the dinner or suooer hour, early eveninqs, Saturday 
morninqs and Sunday afternoons to accommodate the schPf^ul»»s of families in their 
communities. This flexibility of schedulinq, alonq wi<-h the heavy emohasis on 
outreach, most likely accounts for thr increased oartic^oation of fathers, sinqle 
oarents, emoloyed oarents, teen oarents and others who have qenerally not been well 
reoresented in the oast. 

^VlVll- ^^^] collected each year to orovide for local and state comoarisons 

with this information. 
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A PLAN FOR THE COLLECTION OF INFORMATION 
NEEDED FOR MONITORING AND EVALUATION OF PROGRAMS 



The bask data collected on each child and family Is olanned to serve several 
ourooses: 

To identify t^e families and children both for the keeoinq of attendance records 
and to oermit later follow-uo; 

To orovide the oroqran st^ff witn emergency information if it is needed; 

To orovide a description of the comoosition of the family both fur current 
reoortinq ourooses and for the construction of samoles for later study; 

To orovide some very basic family history: its mobility, its changes over time, 
and any soecial characteristics of the family or its members • 

To get information on any soecial history of the child that might have long-term 
effects, e.g., soecial medical conditions, trauma, sensitivities, etc.; 

The family's oast exoerience with ECFE orograms, and the source of their 
referral to the orogram; 

A descriotion of the soecific orogram in which they are oarticioating, its 
meeting times, duration, size and leadershio, so that selective evaluations can 
be made of such things as weekday orograms, different orogram tyoes, etc. 

These seven kinds of information should be gathered wUhin the first month of a 
orogram sequence, with the basic identifying information gathered immediately uoon 
entrance. This latter ooint is imoortant because it could be necessary, in the 
future, to comoare oeoole whw droooed out with oeoole who stayed with the orogram. 

Additionally, it will be imoortant to ask oarents, early in their initial sessions, 
to sign a release to oermit future follow-uo through school and other oublic records. 
In most studies of this kind, oarents who are given exolanations regarding the need 
for follow-uD stuo:ds show little resistance to signing such a release. It is 
imoortant, however, that staff are adequately oreoared to do this. 

The following is a orooosed enrollment form to be used statewide. It would be 
contained on a single oaqe two-oart carbonless form. One cooy would remain in the 
district; the other would be submitted to the state office or other cfrntral olace to 
assure that the information is retained and is accessible a decade or more from now. 

EARLY CHILDHOOD FAMILY EDUCATION PROGRAM ENROLLMENT FORM 

DISTRICT NO RECORD NO^ 



TODAY'S DATE 



CHILD'S NAME 



CHILD'S BIRTHDATE 
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NAME OF CARETAKER 



HOME ADDRESS 



HOUSE APARTMENT OWN RENT 

HOME TELEPHONE OTHER PHONE__ 

FAMILY PHYSICIAN _PHONE 

SPECIAL HEALTH INFO. 

PRIOR ENROLLMENTS IN ECFE^ 

HOW DID YOU HEAR ABOUT THIS PROGRAM? 

SESSIONS YOU WILL ATTEND: MORNING ^AFTERNOON EVENING 

WEEKDAYS W EEKENDS 

ANY SPECIAL CHARACTERISTICS OR UNUSUAL HISTORY OF THIS CHILD 



OTHER PERSONS LIVING WITH THIS CHILD 

'MOULTS M F RELATIONSHIP: 

OCCUPATION (r oom for four adults on form) 

CHILDREN^ ^AGE F RELATIONSHIP 

\room for four children on fornH ~ 

IN HOW MANY DIFFERENT PLACES HAS THIS CHILD LIVED FOR MORE THAN SIX MONTHS? 
CHILD'S SOCIAL SECURITY NO. 



FOR OFFICE USE ONLY 
REASONS FOR ENROLLING: 

ATTENDANCE SUMMARY 



ERIC 



- 25 - 28 



REASON FOR ENROLLING 

Parents will also be asked to comolete this form at the time of enrollment. Numbers 
of resoonses (1-9) will be recorded on enrollment form by staff. 

Parents attend Early Childhood Family Education Programs for many different reasons. 
To helo us make these sessions more useful to you. olease check the three most 
imoortant reasons you have for enrol linq in this oroqram. If any of your reasons are 
not on this list, write them in on Hine 9. 

1) To learn more about child develooment 

2) To take advantage of the health screening service 

3) To meet and discuss common concerns with other oarents 

To qet started on a relationshio with the oublic schools 

5) To learn how to effectively interact with my child 

6) To orevent later oroblems with my child 

7) To qet exoert helo in child rearinq 

8) To be able to borrow the riqht kinds of toys 

9) 
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A PLAN FOR THE LONG-TERM EVALUATION OF THE OUALITY AND 
COST-EFFECTIVENESS OF EARLY CHILDHOOD FAMILY EDUCATION 

The task of desiqninq and carryinq out a lonq-term evaluation of the quality and 
cost-effectiveness of this oroqram involves five distinct initial tasks: 

1) DeveloDinq a standard instrument for the collection of backqround data 
on the oarticioatinq families; 

2) DeveloDinq a standard orocedure for determininq the real costs of the 
Droqram; 

* 

3) Idontifyinq feasible measures of oroqram quality which all the users of 
these evaluations will find useful; 

4) Selectinq an aooroDriate instrument to monitor oarental acceotance of 
the oroqram; 

5) Selectinq and develooinq valid and useful indicators of oroqram 
outcomes. 

Work on the first of these tasks is already underway, and a oreliminary 
instrument will be f e d-tested this sorinq. It will serve as a standard 
enrollment form; it will incoroorate the information needed for reoorts and for 
the staff s direct work with the family; and it will serve as the baseline from 
which to construct samoles for later evaluation tasks. It will also contain the 
information necessary for the conduct of lonq-ranqe fcllow-uo on the children as 
they move throuqh the oublic schools. Desiqn of the forms will take advantaqe 
of modern data orocessinq techniques to assure that the amount of "oaoerwork' 
required to deal with this information will be keot to a minimum, while enablinq 
raoid analyses of the data and ease of information retrieval. 

Determininq the costs of the oroqram will require aqreement amonq the 
oarti ioatinq school districts on the definition and allocation of costs, on the 
trans/ation of services and overhead into costs, and on the definitions of 
oroqram comoonents. In order to carry out valid cost-effectiveness studies, it 
win be necessary to collect much finer-qrained fiscal information than is 
required for standard audit orocedures. Furthermore, it will be necessary to 
allocate such costs as soace and utilities which are not usually assiqned to 
soecific orograiiis within a school buildinq. 

While the data necessary for such measures of cost-effectiveness are orobably in 
the files of individual districts, it will b- necessary to brinq their business 
manaqers toqether and reach aqreement witi. them on a standard system of 
reoortinq costs that will not overburden them or duolicate cost-control or local 
reoortinq orocedures they already have, and to obtain their acceotance of and 
collaboration in this task. A major reason there are so few measures of the 
cost-effectiveness of oublic service oroqrams has been the difficulty of 
arrivinq at common orocedures and data collection across a larqe number of 
indeoendent orqanizations. Because the validity of these data is so imoortant. 
simoly requirinq" the data will not assure its quality. Furthermore, district 
business manaqers are the best source of consultation to the state as to the 
most reliable and valid measures to use in determininq oroqram costs. 
Preliminary examination of fiscal data recordinq and reoortinq oractices will 
beqin this sorinq. and will be used to identify the areas which require 
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aqreements, definitions, and uniform orocedures. State UFARS and Data 
Acquisition staff will be consulted in constructinq a uniform cost-collection 
system. 

The third task, develooinq a set of useful indicators of oroqram quality, has 
already been aooroached in earlier evaluations conducted under the ausoices of 
the Minnesota Council on Quality Education. These criteria were orlncloally 
defined from the oersoective of staff and oarents, and were determined by the 
judgments of oarticioants and/or observers. These earlier efforts are "beinq 
reviewed with two additional questions in mind: 

1) Are these indicators of quality aoorooriate for audiences other than the 
direct oarticioants in the oroqram? Will we need to develoo additional or 
different indicators to meet the concerns of the qeneral oublic or the 
Leqisiature? 

2) Can we find indicators of quality that do not deoend uoon the fudqments 
of oarticioants or intermittent observers? For examole, are there 
behavioral chanqes in oroqram oarticioants that can be oblectively 
measured? Are there short-term outcomes that differentiate' oroqram 
I^Jm^ ^^^""^ some basic standards of quality that can be defined 
«!5I kJJ D;-?<"'am s ooerations? Can we construct feedback mechanisms that 
serve both self-evaluation and an objective indicator of quality? 

pS?on!!I""n"*' °^ ^'''^^ *° these questions of 

^J^n.i?; Jr*"!??* K^^'"' resources based uoon exoerience 

JitJrf.mI^;c . M ^ ^ necessary to construct ard validate many o' our own 
instruments and orocedures. 

The selection of an aoorooriate Instrument for monitorinq oarental acceotance of 
the oroqram throuqhout the state will be accomolished soon. Durinq orevious 
evaluations, a number of such instruments were develooed. These will be 
examined with a view toward selectinq the one which best combines simolicity of 
use with reliability of measurement. Because oarental acceotance is crucial to 
this oroqram, it will also be necessary to devise a means of solicitinq oarental 
suggestions for oroqram imorovement, and to develoo orocedures to assure 
continuing orogram resoonsiveness to oarental concerns, which will most likely 
change over time. The olan is to continue to Involve oarents as direct 
oarticioants in the selection of the contents of the evaluation orocedures 
t.iemselves. because they are, in many ways, the most relevant judges of the 
orogram and its orincioal audience. 

The selection of feasible and useful indicators of long-term oroqram outcomPS 
will be a comolex task. It is not reasonable to assume that this orooram is thP 
most imoortant Influence in a child's life, and seoaratinq its effects from all 
the other influences in the lives of families and children requires care in both 
the analytic orocedures and in the selection of outcomes to be studied. The 
outcomes to be studied must meet a variety of criteria. 

They must be objectively and relia bly measureable. An outcome indicator which 
requires a teacner's jufiqment, or is subject to the vaoaries of the moment, is 
not credible or useful. 

They must be feasible of measurement over larqe oooulations. A measure which is 
exoensive ot time or equloment is not usei^ul l^or this ouroose.- For examole. 
individual diaqnostic studies of children may be both reliable and objective. 
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but are not feasible for evaluating a statewide oroqram. While small samoles 
may be studied Intensively for soeclfic evaluative ourooses, such orocedures 
cannot meet the state's major needs In assessing the worth of this Investment. 

They must sensibly reflect the ourDOses of the program. It does not serve the 
state's needs to have measures of outcomes that are not relevant to the central 
ourooses of the program, regardless of how objective or feasible they are. An 
examole of mis-selection of an outcome measure was the attempt to evaluate Hejd 
Start by the use of 10 measures. Since Head Start was never Intended to affect 
10, the use of such a measure, regardless of the outcome, was not only 
Irrelevant to the Program's purpose, but, by drawing attention away from the 
principal purpose, had mischievous effects on the program. Too often an 
Instnjment Is selected for evaluation because It exists rather than because it 
Is a valid measure of the program Itself. In the case of this program, there Is 
unusual clarity and concensus about the purposes of the program. This clarity 
will hfelp In selecting outcome measures which all of the audiences concerned 
with the program can agree will be sensible indicators of Program success. Such 
indicators should be behavioral, and not simply scores on tests. 

Similarly, the measures selected should make sense within the Minnesota public 
school environment. Labelled as ecological validity, this principle reguires 
that a measure or criterion be appropriate to the setting in which it is used. 



The Program outcome measures must be useful. The effort Involved in 
longitudinal research is ;uch that it should not be sufficient to be able to 
report that a program was or was not effective in meeting the stated goals. It 
should also provide information which will enable program Improvement, which 
will indicate areas of relative strength, identify characteristics of families 
or persons most likely to benefit from the orograr, and indicate the ways in 
^ which this Program relates to other prograrrs and e-oeriences. The evaluative 

^ detail should also correspond to the detail with which program costs are 

recorded, so that the relative co?t, as well as thp relative effectiveness of 
m separate components of the program, can be evaluated along with the overall 

costs and effects. 

It is important for the Legislature to recognize that the long-term effects of 
this program are likely to be the most important effects, and that the phrase 
"long-term" means exactly that. One cannot expect long-term findings until the 
children in the first round of programs have graduated from secondary school. 
Thus we will be collecting information for use fifteen and more years from now, 
and need to set up data collection systems which will enable us to retrieve 
information covering the whole public school career of these children, as well 
es be able to find them and thei*- parents two decades from now. 

While work will begin on the first four of these five tasks within the year, the 
planning and development for the fifth task wiVi take longer. However, since we 
will not need to collect those measures for some time to come, and will have 
basic information on each family from the time of enrollment, we can afford the 
time to meet the challenger* of the fifth task sensibly and carefully. We exoect 
to get assistance fn that task from parents and staff, from school personnel, 
from experts at Minnesota colleges and universities, and from consultants and 
the research literature. 



• 
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We will be concerned with answerlnq the followlnq kinds of lonq-term questions: 

- Who benefits, and how, from this oroqram? 

- What are the relative costs and outcomes of various oroqram comoonents? 

- Does oartlcloatlon In this oroqram affect the stability of family life? 

- Are the children of oartldoants less likely to oresent behavioral 
difficulties In the classroom or elsewhere? 

- Are oarent oartldoants more likely to remain active In school affairs 
throuqhout their children's school careers? 

- Are the children of oartlcloants more likely to finish secondary school? 

This list of questions Is not meant to be exhaustive; It Is Illustrative of the 
kinds of outcomes considered sensible to seek, useful to learn, and feasible of 
measurement. 
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